gh-140009: Optimize JSON parsing with object_pairs_hook via PyTuple_FromArray - #144772

Summary

This PR optimizes the object_pairs_hook path in Modules/_json.c (e.g., used by json.loads(s, object_pairs_hook=list)) by replacing PyTuple_Pack with PyTuple_FromArray.

PyTuple_Pack processes arguments variadically via va_list, while PyTuple_FromArray performs a direct memcpy from a stack-allocated array. For a fixed-size-2 tuple constructed on every key-value pair in the hot path, this eliminates unnecessary overhead.

Benchmarks (PGO+LTO)

Validated using pyperf in --rigorous mode on a full production build. Results were reproduced across multiple independent sessions.

• Platform: macOS arm64 (Apple M-series)
• Build: --enable-optimizations --with-lto
• Tool: pyperf (--rigorous mode)
• Baseline: upstream/main
• Candidate: pytuple-json-objectpairs-fromarray (92d3f1a)

Geometric mean: 1.01x faster

# Target: parsing with object_pairs_hook=list
python -m pyperf command --rigorous --name json_pairs_hook_dense
  ./python.exe bench_json_pairs_hook.py pairs 320 64 64

# Control: standard parsing (no hook)
python -m pyperf command --rigorous --name json_pairs_hook_control
  ./python.exe bench_json_pairs_hook.py control 320 64 64

import json
import sys

def build_payload(objects, fields):
    obj = "{" + ",".join(f'"k{i}":{i}' for i in range(fields)) + "}"
    return "[" + ",".join(obj for _ in range(objects)) + "]"

def main():
    mode, loops, objects, fields = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])
    payload = build_payload(objects, fields)

    for _ in range(loops):
        if mode == "pairs":
            json.loads(payload, object_pairs_hook=list)
        else:
            json.loads(payload)

if __name__ == "__main__":
    main()

Analysis

The json_pairs_hook_dense benchmark parses a large JSON array of objects using object_pairs_hook=list. In this scenario, every key-value pair requires a size-2 tuple. The switch to the array-based API yields a ~2% speedup on this specific codepath, with a conservative geometric mean of ~1% across both benchmarks.

Notably, the candidate also shows a significant reduction in variance (±9 ms → ±3 ms), suggesting more deterministic performance on the optimized path.

The json_pairs_hook_control case confirms that the standard JSON decoding path (using the default decoder) is unaffected.

• Issue: Improve performance by replacing PyTuple_Pack with PyTuple_FromArray #140009